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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified atxive is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )S Responsive to communication(s) filed on 05 January 2004 . 
2a)Kl This action is FINAL. 2b)n This action is non-final. 

3) D Since this application is in condition for allowance except for fornnal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) K Claim(s) 1^7 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) K Claim(s) 7 is/are allowed. 

6) S Claim(s) is/are rejected. 

Claim(s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) \3 The specification is objected to by the Examiner. 

10)0 The drawing(s) filed on is/are: a)n accepted or b)n objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 119 

12)n Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (0- 
a)n All b)n Some * 0)0 None of: 

1 0 Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 



Claim Rejections - 35 USC § 103 



The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

1 . Claims 1-6 are rejected under 35 U.S.C. 103(a) as being unpatentable over Kirksey (US 
Patent No. 5,938,447 in view of Schulz (US Patent No. 6,185,538), in further view of Morioka et 
al (US Patent No. 6,226,443), hereinafter referred to as Morioka. 

Kirksey discloses a method and system for creating an audio-visual work, which displays 
the text version of the words heard in the audio. 

2. Regarding claim 1, at col. 5, lines 49-51, Kirksey discloses an embodiment of the system 
in which words maybe placed on an audiovisual work so that they are only visible to a viewer 
who uses a special reading device, which reads on "special reproducing method for specially 
reproducing sound by using an information reproducing apparatus for reading out data from a 
recording medium having audio data and video data, a special reproducing method of sound." 

In the abstract, Kirksey discloses that the system displays the text version of the words 
heard in the audio. However, Kirksey does not specifically disclose the audio data is read out 
during special reproduction is converted to text via sound recognition. However, speech to text 
conversion of audio data is well known in the art of speech recognition. 

In a similar field of endeavor, Schulz discloses a system for editing digital video and 
audio information, which specifically provides for the recording and reproducing video and 



Application/Control Number: 09/615,313 Page 3 

Art Unit: 2654 

audio information and the conversion of the audio source information into text via a speech 
recognition module (col. 4, lines 25-33), which reads on "the audio data read out from a 
recording medium during special reproduction is converted into text data by sound recognition." 
Schulz teaches that the system is advantageous in obtaining recognized text of different voices 
(col. 3, lines 15-23) and foreign languages (col. 4, line 58 continuing to col. 6, line 6). 

Therefore, it would have been obvious to one of ordinary skill at the time of the invention 
to modify the system Kirksey to implement speech recognition of the audio information as taught 
by Schulz, for the purpose of obtaining the corresponding text of foreign language audio, as 
suggested by Schulz. 

Kirksey does not specifically teach implementation of audio data read out from a 
recording medium during n-speed reproducing, wherein n is a positive number greater than 1 . 
However, implementation data read out from a recording medium during of n-speed reproducing, 
where n is a positive number greater than 1 was well known in the art. 

In a similar field of endeavor, Morioka teaches a recording and reproducing apparatus for 
recording and reproducing hybrid data, including video data, audio data, and additional data, 
such as text, such that the system reproduces and outputs all the forms of data. Specifically, at 
col. 15, line 40 to col. 16, line 60, Morioka teaches that during reproduction of recorded 
information, the system reproduces recorded information at an equal rate or a different rate 
to/from the rate during recording, and that the rate of the signal recorded is converted into the 
desired reproducing rate, which reads on "contents of the sound which are recorded on the 
recording medium for n seconds are displayed during 1 second of the specially reproduced 
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images." Morioka teaches the system is advantageous because it considerably improves editing 
efficiency (col. 5, lines 56-59). 

Therefore, it would have been obvious to one of ordinary skill at the time of the invention 
to modify the system of Kirksey to implement n-speed reproducing, wherein n is a positive 
number greater than 1 , as taught by Morioka, for the purpose of improving editing efficiency, as 
also suggested by Morioka. 

Additionally, at col. 5, lines 56-65, Kirksey discloses the words corresponding to the 
audio are displayed as an integral part of and superimposed on the pictorial scene of the work 
and at col. 9, lines 31-37, Kirksey discloses altering the sound reproduction for altering the 
display time of corresponding text, which reads on "and the characters representing text data are 
displayed, being overlapped with special reproduced images" and "contents of the sound are 
displayed during 1 second of the specially reproduced images." 

Regarding claim 2, Kirksey, Morioka, and Schulz teach everything as claimed in claim 1. 
Additionally, at col. 9, lines 31-37, Kirksey discloses altering the sound reproduction for altering 
the display time of corresponding text, which reads on "contents of sounds for n-seconds 
recorded on the recording medium are displayed for one second." 

Regarding claim 3, at col. 5, lines 49-51, Kirksey discloses an embodiment of the system 
in which words maybe placed on an audiovisual work so that they are only visible to a viewer 
who uses a special reading device, which reads on "special reproducing apparatus for reading out 
data fi-om a recording medium having audio data and video data." 

In the abstract, Kirksey discloses that the system displays the text version of the words 
heard in the audio. However, Kirksey does not specifically disclose the audio data is read out 
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during special reproduction is converted to text via sound recognition. However, speech to text 
conversion of audio data is well known in the art of speech recognition. 

In a similar field of endeavor, Schulz discloses a system for editing digital video and 
audio information, which specifically provides for the recording and reproducing video and 
audio information and the conversion of the audio source information into text via a speech 
recognition module (col. 4, lines 25-33), which reads on "the audio data read out fi-om a 
recording medium during special reproduction is converted into text data by sound recognition." 
Schulz teaches that the system is advantageous in obtaining recognized text of different voices 
(col. 3, lines 15-23) and foreign languages (col. 4, line 58 continuing to col. 6, line 6). 

Therefore, it would have been obvious to one of ordinary skill at the time of the invention 
to modify the system Kirksey to implement speech recognition of the audio information as taught 
by Schulz, for the purpose of obtaining the corresponding text of foreign language audio, as 
suggested by Schulz. 

Kirksey does not specifically teach implementation of audio data read out fi"om a 
recording medium during n-speed reproducing, wherein n is a positive number greater than 1 . 
However, implementation data read out from a recording medium during of n-speed reproducing, 
where n is a positive number greater than 1 was well known in the art. 

In a similar field of endeavor, Morioka teaches a recording and reproducing apparatus for 
recording and reproducing hybrid data, including video data, audio data, and additional data, 
such as text, such that the system reproduces and outputs all the forms of data. Specifically, at 
col. 15, line 40 to col. 16, line 60, Morioka teaches that during reproduction of recorded 
information, the system reproduces recorded information at an equal rate or a different rate 
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to/from the rate during recording, and that the rate' of the signal recorded is converted into the 
desired reproducing rate, which reads on "contents of the sound which are recorded on the 
recording medium for n seconds are displayed during 1 second of the specially reproduced 
images." Morioka teaches the system is advantageous because it considerably improves editing 
efficiency (col. 5, lines 56-59). 

Therefore, it would have been obvious to one of ordinary skill at the time of the invention 
to modify the system of Kirksey to implement n-speed reproducing, wherein n is a positive 
number greater than 1, as taught by Morioka, for the purpose of improving editing efficiency, as 
also suggested by Morioka. 

Additionally, at col. 5, lines 56-65, Kirksey discloses the words corresponding to the 
audio are displayed as an integral part of and superimposed on the pictorial scene of the work 
and at col. 9, lines 31-37, Kirksey discloses altering the sound reproduction for altering the 
display time of corresponding text, which reads on "and the characters representing text data are 
displayed, superimposed on special reproduced images displayed in a display device" and 
"contents of the sound are displayed during 1 second of the specially reproduced images." 

Regarding claim 4, Kirksey, Morioka, and Schulz teach everything as claimed in claim 3. 
Additionally, at col. 5, lines 49-51, Kirksey discloses an embodiment of the system in which 
words maybe placed on an audiovisual work so that they are only visible to a viewer who uses a 
special reading device, which reads on "reading means for reading out audio data and video data 
fi-om a recording medium." 



Application/Control Number: 09/61 5,3 1 3 Page 7 

Art Unit: 2654 

At col. 5, lines 56-65, Kirksey discloses the words corresponding to the audio are 
displayed as an integral part of and superimposed on the pictorial scene of the work, which reads 
on "display means for displaying reproduced images based on video data on the display device." 

In the abstract, Kirksey discloses that the system displays the text version of the words 
heard in the audio. However, Kirksey does not specifically disclose the audio data is read out 
during special reproduction is converted to text via sound recognition. However, speech to text 
conversion of audio data is well known in the art of speech recognition. 

Schulz discloses a system for editing digital video and audio information, which 
specifically provides for the recording and reproducing video and audio information and the 
conversion of the audio source information into text via a speech recognition module (col. 4, 
lines 25-33), which reads on "the audio data read out fi-om a recording medium during special, 
reproduction is converted into text data by sound recognition." Schulz teaches that the system is 
advantageous in obtaining recognized text of different voices (col. 3, lines 15-23) and foreign 
languages (col. 4, line 58 continuing to col. 6, line 6). 

Therefore, it would have been obvious to one of ordinary skill at the time of the invention 
to modify the system Kirksey to implement speech recognition of the audio information as taught 
by Schulz, for the purpose of obtaining the corresponding text of foreign language audio, as 
suggested by Schulz. 

At coL 5, lines 56-65, Kirksey discloses the words corresponding to the audio are 
displayed as an integral part of and superimposed on the pictorial scene of the work, which reads 
on "and the characters representing text data are displayed, being overlapped with special 
reproduced images displayed in a display device" 
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At col. 8, lines 25-39, Kirksey discloses the apparatus for creating the audiovisual work 
which includes an operator station, video text generator, audiovisual input means, and optical 
manipulator for combining text and audiovisual work, which reads on "control means for 
instructing reading means, display means and conversion means to perform special 
reproduction." 

Regarding claim 5, Kirksey, Morioka, and Schulz teach everything as claimed in claim 1. 
Additionally, at col. 9, lines 31-37, Kirksey discloses altering the sound reproduction for altering 
the display time of corresponding text, which reads on "contents of sounds for n-seconds 
recorded on the recording medium are displayed for one second." 

Regarding claim 6, at col. 5, lines 49-51, Kirksey discloses an embodiment of the system 
in which words maybe placed on an audiovisual work so that they are only visible to a viewer 
who uses a special reading device, which reads on "special reproducing apparatus for reading out 
data from a recording medium having audio data and video data." 

In the abstract, Kirksey discloses that the system displays the text version of the words 
heard in the audio. However, Kirksey does not specifically disclose the audio data is read out 
during special reproduction is converted to text via sound recognition. However, speech to text 
conversion of audio data is well known in the art of speech recognition. 

In a similar field of endeavor, Schulz discloses a system for editing digital video and 
audio information, which specifically provides for the recording and reproducing video and 
audio information and the conversion of the audio source information into text via a speech 
recognition module (col. 4, lines 25-33), which reads on "the audio data read out from a 
recording medium during special reproduction is converted into text data by sound recognition." 
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Schulz teaches that the system is advantageous in obtaining recognized text of different voices 
(col. 3, lines 15-23) and foreign languages (coL 4, line 58 continuing to col. 6, Une 6). 

Therefore, it would have been obvious to one of ordinary skill at the time of the invention 
to modify the system Kirksey to implement speech recognition of the audio information as taught 
by Schulz, for the purpose of obtaining the corresponding text of foreign language audio, as 
suggested by Schulz. 

Kirksey does not specifically teach implementation of audio data read out fi"om a 
recording medium during n-speed reproducing, wherein n is a positive number greater than 1 or a 
system controller for controlling a reproducing speed of the recording medium. However, 
implementation of a system controller for controlling a reproducing speed of a recording medium 
and data read out from a recording medium during of n-speed reproducing, where n is a positive 
number greater than 1 , was well known in the art. 

In a similar field of endeavor, Morioka teaches a recording and reproducing apparatus for 
recording and reproducing hybrid data, including video data, audio data, and additional data, 
such as text, such that the system reproduces and outputs all the forms of data. Specifically, at 
col. 15, line 40 to col. 16, line 60, Morioka teaches that during reproduction of recorded 
information, the system reproduces recorded information at an equal rate or a different rate 
to/fi"om the rate during recording, and that the rate of the signal recorded is converted into the 
desired reproducing rate, which reads on "contents of the sound which are recorded on the 
recording medium for n seconds are displayed for 1 second." Morioka teaches the system is 
advantageous because it considerably improves editing efficiency (col. 5, lines 56-59). 
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Therefore, it would have been obvious to one of ordinary skill at the time of the invention 
to modify the system of Kirksey to implement n-speed reproducing, v^herein n is a positive 
number greater than 1, as taught by Morioka, for the purpose of improving editing efficiency, as 
also suggested by Morioka. 

Additionally, at col. 5, lines 56-65, Kirksey discloses the words corresponding to the 
audio are displayed as an integral part of and superimposed on the pictorial scene of the work 
and at col. 9, lines 31-37, Kirksey discloses altering the sound reproduction for altering the 
display time of corresponding text, which reads on "and the characters representing text data are 
displayed, being overlapped with special reproduced images" and "contents of the sound are 
displayed for 1 second " 

Kirksey does riot specifically teach MPEG audio and video decoders for decoding audio 
data and video data. However, implementation of MPEG decoders was well known in the art. 

Morioka teaches implementation of MPEG decoders at col. 12, line 39 continuing to col. 
13, line 13. 

Therefore, it would have been obvious to one of ordinary skill at the time of the invention 
to modify the system of Kirksey to implement MPEG decoding as taught by Morioka, for the 
purpose of displaying text or pertinent words with the images of a perceptually encoded audio- 
visual work. 

Response to Arguments 
3. Applicant's arguments filed January 5, 2004, have been fully considered but they 
are not persuasive. Applicant argues that the combination of elements and steps set forth in 
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independent claims 1,3, and 6 are not disclosed or made obvious by the prior art of Kirksey, 
Schulz, and Morioka et al. Applicant argues Morioka is not concerned with displaying contents 
of recorded audio data, and therefore does not teach or suggest displaying the contents of audio 
data for n-seconds recorded on the recording medium for 1 second. In response to applicant's 
arguments against the references individually, one cannot show nonobviousness by attacking 
references individually where the rejections are based on combinations of references. See In re 
Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 
USPQ 375 (Fed. Cir. 1986). In this instance, Kirksey was cited as disclosing an embodiment of 
the system in which words maybe placed on an audiovisual work so that they are only visible to 
a viewer who uses a special reading device; the words corresponding to the audio are displayed 
as an integral part of and superimposed on the pictorial scene of the work and altering the sound 
reproduction for altering the display time of corresponding text; Schulz was cited as disclosing s 
a system for editing digital video and audio information, which specifically provides for the 
recording and reproducing video and audio information and the conversion of the audio source 
information into text via a speech recognition module; and Morioka was cited for teaching 
during reproduction of recorded information, the system reproduces recorded information at an 
equal rate or a different rate to/from the rate during recording, and that the rate of the signal 
recorded is converted into the desired reproducing rate. Therefore, the combination of Kirksey, 
Schulz, and Morioka provide support for the limitations. 
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Allowable Subject Matter 



4. Claim 7 is allowed. 

The following is a statement of reasons for the indication of allowable subject matter: 
The prior art of record fails to specifically teach or suggest the combination of an information 
reproducing apparatus for reading out data fi'om a recording medium having audio data and 
video data, comprising, inter alia, a data analysis processing unit for analyzing the audio data 
according to speed change information fi-om the system controller and for improving the 
accuracy of sound recognition by suppressing unnecessary noise; a data table for registering the 
text data and the corresponding audio data; and a data conversion processing unit for integrating 
the timing of the audio data from the data analysis processing unit with the timing of the audio 
data from the data table, and searching audio data fi-om the data table nearest to the audio data 
fi'om the data analysis processing unit by comparing each audio data and receiving the text data 
corresponding to the audio data from the data table. 



5. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
poUcy as set forth in 37 CFR 1 . 1 36(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 



Conclusion 
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CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 



6. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Angela A. Armstrong whose telephone number is 703-308-6258. 
The examiner can normally be reached on Monday-Thursday 7:30-5:00 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Richemond Dorvil can be reached on (703) 305-9645. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 

Angela A. Armstrong 

Examiner 

Art Unit 2654 



AAA 

March 18, 2004 



